Reinforcement learning approaches for the parallel machines job shop scheduling problem

This paper addresses the application of AI techniques in a practical OR problem , i.e. scheduling. Scheduling is a scientific domain concerning the allocation of tasks to a limited set of resources over time. The goal of scheduling is to maximize (or minimize) different optimization criteria such as the makespan (i.e. the completion time of the last operation in the schedule), the occupation rate of a machine or the total tardiness. In this area, the scientific community usually classifies the problems according to the characteristics of the systems studied. Important characteristics are: the number of machines available (one machine, parallel machines), the shop type (Job Shop, Open Shop or Flow Shop), the job characteristics (such as pre-emption allowed or not, equal processing times or not) and so on [1]. In this work we present two Reinforcement Learning approaches for the Parallel Machines Job Shop Scheduling Problem (JSP-PM). The job-shop scheduling problem with parallel machines also known as the flexible job shop scheduling problem, represents an important problem encountered in current practice of manufacturing scheduling systems. It consists of assigning any operation of each job to a resource, i.e. one of the machines in a pool of identical parallel machines, in order to minimize a certain objective [2]. The pool of identical parallel machines, is sometimes called a machine type, a workcenter or also a flexible manufacturing cell [2]. The difference with the classic Job-Shop (JSSP) is that instead of having a single resource for each machine type, in flexible manufacturing systems a number of parallel machines are available in order to both increase the throughput rate and avoid production stop when machines fail or maintenance occurs. The objective we consider here is the minimization of the schedule makespan. Literature on job shop scheduling with parallel machines is not rare, but approaches using learning based methods are. In the literature we find different (meta-)heuristic approaches for this problem. In [3] a tabu-search method which was originally introduced for the classic JSSP, is applied. In [4] a variable neighborhood genetic algorithm is used and in [2] a hybrid method combining a genetic algorithm and an ant colony optimization method is proposed. We will use the latter reference to compare our results with. Reinforcement Learning is the problem faced by an agent that must learn behavior through trial-and-error interactions with a dynamic environment. Each time the agent performs an action in its environment, …